Method and apparatus for controlling vehicular user interface under driving condition

ABSTRACT

A method for controlling an in-vehicle user interface includes: providing a vehicle window that serves as an input/output module engaged with at least one of a computing device, a camera and a multimedia device; monitoring whether a change of an air pressure at a neighboring area of the vehicle window is beyond a predetermined threshold range; sensing a touch input via the vehicle window; and generating, in response to the change of the air pressure and the touch input, a control signal of the input/output module.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims under 35 U.S.C. § 119(a) the benefit of KoreanPatent Application No. 10-2016-0128545, filed on Oct. 5, 2016 in theKorean Intellectual Property Office, the entire contents of which areincorporated by reference herein.

BACKGROUND (a) Technical Field

The present disclosure relates to a method and apparatus for controllingan in-vehicle user interface under a driving condition or an environmentof a vehicle, and more particularly, to a control method and apparatusfor enabling a driver or a passenger to use a vehicle window as a userinterface of a computing device according to a driving condition or anenvironment of the vehicle.

(b) Description of the Related Art

Vehicle users typically have an interest in operating variousinstruments and/or accessories while traveling in a vehicle. However,user interfaces (e.g., keypad, screen, or etc.) that can be provided inthe vehicle may be spatially limited. Further, adding a complex userinterface in the vehicle may impair a driving safety of a driver or apassenger (i.e., an occupant).

A vehicle includes a window. One or more windows mounted on the vehicleare essential for a driver or a user to ensure visibility for the front,rear, and side views while the vehicle moves. If such a window could beused as a user interface, the convenience of the driver or the usercould be enhanced.

Recently, a smart window technology has been suggested. For example, adriver or a user adjusts the amount of sunlight flowing through thewindshield of the vehicle by simple manipulation. Such a smart windowtechnology uses a special film inserted in a car window to change thetransparency of the car widow based on a received electrical controlsignal.

On the other hand, a vehicle windshield, which not merely secures adriver's view, but serves as a barrier to wind and rain, can include a‘smart window’ function with cutting-edge electronic technology. Forexample, a head-up display (HUD) may be applied to an in-vehicle glass,which can achieve or add functions/characteristics such as an automaticlighting control, an ultra-light weight, and a touch screen.

The in-vehicle head-up display (HUD) is a technology that projectsinformation or guidance, which is displayed on a conventional instrumentpanel such as a dashboard, on the eye level of the driver's window. Thistechnique has been used to expand a pilot's front view in a militarycombat plane. But, the head-up display (HUD) also has been recentlyapplied to vehicles so that the driver or user can obtain necessaryinformation without having to look at the instrument panel or navigationdevice while driving. As a result, a driver's or user's concentrationwhile driving can be improved and driving safety can be enhanced.Recently, the head-up display (HUD) for a vehicle can engage with afront camera, a Global Positioning System (GPS), and various sensors soas to display different types of information associated with driving.

SUMMARY

The present disclosure provides a method and apparatus for controlling avehicle window as an input/output module such as a user interface for atleast one of: a computing device, a camera, and a multimedia device,which can be equipped in or engaged with a vehicle according to adriving state/condition and a driving mode of the vehicle.

Further, the disclosure provides a control device and a control methodin which a vehicle window is as a transparent touch screen for ensuringthe visibility of a driver, a user, a passenger, or an occupant.

In addition, the disclosure provides a control method and a controlmethod for using an air pressure sensor to initialize or reset a vehiclewindow serving as a transparent touch screen.

Further, the disclosure provides a control device and a control methodcapable of capturing, editing, or processing information of scenery orobjects that a driver, a user, a passenger or an occupant can viewthrough a vehicle window.

A method for controlling an in-vehicle user interface can includeproviding a vehicle window serving as input/output module engaged withat least one of a computing device, a camera and a multimedia device;monitoring, by an air pressure sensor, whether a change of an airpressure at a neighboring area of the vehicle window is beyond apredetermined threshold range; sensing a touch input via the vehiclewindow; and generating, in response to the change of the air pressureand the touch input, a control signal of the input/output module.

The predetermined threshold range can be set within a pressure range ofbreathing of a person.

The input/output module can include a touch input device and a displayor a screen used for the at least one of the computing device, thecamera and the multimedia device.

The generating the control signal can include activating the touch inputdevice and the display or the screen in response to the change of theair pressure; and deactivating the touch input device and the display orthe screen when the touch input is not entered for a predetermined timeafter the touch input device and the display or the screen areactivated.

The generating the control signal can further include generating, inresponse to the change of the air pressure, a signal for resetting thetouch input device and the display or the screen.

The method can further include showing, in response to the change of theair pressure, a first touch input button, used for selecting one of thecomputing device, the camera and the multimedia device, on the vehiclewindow; and showing, in response to user's input via the first touchinput button, a second touch input button, used for operating orcontrolling selected one of the computing device, the camera and themultimedia device, on the vehicle window.

The second touch input button, when the camera is selected via the firsttouch input button, can include a frame determining a photography areaof scenery or an object through the vehicle window; a shutter buttontaking a photograph or a video through the frame; and an editing buttonediting the photograph or the video.

The method can further include engaging the at least one of thecomputing device, the camera and the multimedia device with anaudio-video-navigation device equipped in a vehicle; and engaging the atleast one of the computing device, the camera and the multimedia devicewith a mobile device coupled with a wireless communication unit equippedin the vehicle.

The method can further include deactivating the vehicle window adjacentto driver's seat when movement of a vehicle is detected by a wheel speedsensor; and enabling an air pressure sensor detecting the change of theair pressure when a vehicle door and the vehicle window are closed.

An apparatus for controlling a window mounted on a vehicle, equipped orengaged with at least one of a computing device, a camera, and amultimedia device, can include a processing system that comprises atleast one data processor and at least one computer-readable memorystoring a computer program. Herein, the processing system is configuredto cause the apparatus to provide a vehicle window serving as aninput/output module engaged with at least one of a computing device, acamera and a multimedia device; monitor, by an air pressure sensor,whether a change of an air pressure at a neighboring area of the vehiclewindow is beyond a predetermined threshold range; sense a touch input,by a touch input unit, via the vehicle window; and generate, by acontroller, in response to the change of the air pressure and the touchinput, a control signal of the input/output module.

An apparatus for controlling an in-vehicle user interface can include acontroller configured to provide a vehicle window serving as aninput/output module engaged with at least one of a computing device, acamera and a multimedia device; an air pressure sensor configured tomonitor whether a change of an air pressure at a neighboring area of thevehicle window is beyond a predetermined threshold range; and a touchinput unit configured to sense a touch input via the vehicle window.Herein, the controller generates a control signal of the input/outputmodule in response to the change of the air pressure and the touch inputdelivered from the air pressure sensor and the touch input unit.

The predetermined threshold range can be set within a pressure range ofbreathing of a person.

The input/output module can include a touch input device and a displayor a screen used for the at least one of the computing device, thecamera and the multimedia device.

The controller can activate the touch input device and the display orthe screen in response to the change of the air pressure, whiledeactivating the touch input device and the display or the screen whenthe touch input is not entered for a predetermined time after the touchinput device and the display or the screen are activated.

The controller can generate, in response to the change of the airpressure, a signal for resetting the touch input device and the displayor the screen.

The controller can show, in response to the change of the air pressure,a first touch input button, used for selecting one of the computingdevice, the camera and the multimedia device, on the vehicle window,while showing, in response to user's input via the first touch inputbutton, a second touch input button, used for operating or controllingselected one of the computing device, the camera and the multimediadevice, on the vehicle window.

The apparatus can further include a communication unit configured toengage the at least one of the computing device, the camera and themultimedia device with an audio-video-navigation device equipped in avehicle via a controller area network; and engage the at least one ofthe computing device, the camera and the multimedia device with a mobiledevice via a short-range wireless communication method.

The air pressure sensor can be arranged at a surrounding area of thevehicle window, while the camera can be arranged in a vehicle windowframe.

The apparatus can further include a wheel speed sensor configured todetect movement of a vehicle. Herein, the controller deactivates thevehicle window adjacent to driver's seat when the wheel speed sensordetects movement of the vehicle.

The apparatus can further include a door sensor configured to detectopening or closing of a vehicle door; and a window sensor configured todetect opening or closing of the vehicle window. Herein, the controllercan enable the air pressure sensor detecting the change of the airpressure when the vehicle door and the vehicle window are closed.

Advantages, objects, and features of the disclosure will be set forth inpart in the description which follows and in part will become apparentto those having ordinary skill in the art upon examination of thefollowing or may be learned from practice of the disclosure. Theobjectives and other advantages of the disclosure may be realized andattained by the structure particularly pointed out in the writtendescription and claims hereof as well as the appended drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the disclosure and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the disclosure andtogether with the description serve to explain the principle of thedisclosure. In the drawings:

FIG. 1 is a flowchart depicting a control method for an in-vehicle userinterface;

FIG. 2 is a schematic view of a vehicle including vehicle window(s) usedas a user interface;

FIG. 3 is a schematic view depicting an operation of an air pressuresensor;

FIG. 4 illustrates a schematic view depicting a first example of anin-vehicle user interface;

FIG. 5 illustrates a schematic view depicting a second example of thein-vehicle user interface;

FIG. 6 illustrates a schematic view depicting a third example of thein-vehicle user interface; and

FIG. 7 is a block diagram of a control apparatus for an in-vehicle userinterface.

DETAILED DESCRIPTION

It is understood that the term “vehicle” or “vehicular” or other similarterm as used herein is inclusive of motor vehicles in general such aspassenger automobiles including sports utility vehicles (SUV), buses,trucks, various commercial vehicles, watercraft including a variety ofboats and ships, aircraft, and the like, and includes hybrid vehicles,electric vehicles, plug-in hybrid electric vehicles, hydrogen-poweredvehicles and other alternative fuel vehicles (e.g. fuels derived fromresources other than petroleum). As referred to herein, a hybrid vehicleis a vehicle that has two or more sources of power, for example bothgasoline-powered and electric-powered vehicles.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the disclosure.As used herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. As used herein, the term “and/or”includes any and all combinations of one or more of the associatedlisted items. Throughout the specification, unless explicitly describedto the contrary, the word “comprise” and variations such as “comprises”or “comprising” will be understood to imply the inclusion of statedelements but not the exclusion of any other elements. In addition, theterms “unit”, “-er”, “-or”, and “module” described in the specificationmean units for processing at least one function and operation, and canbe implemented by hardware components or software components andcombinations thereof.

Further, the control logic of the present disclosure may be embodied asnon-transitory computer readable media on a computer readable mediumcontaining executable program instructions executed by a processor,controller or the like. Examples of computer readable media include, butare not limited to, ROM, RAM, compact disc (CD)-ROMs, magnetic tapes,floppy disks, flash drives, smart cards and optical data storagedevices. The computer readable medium can also be distributed in networkcoupled computer systems so that the computer readable media is storedand executed in a distributed fashion, e.g., by a telematics server or aController Area Network (CAN). In the description of the disclosure,certain detailed explanations of related art are omitted when it isdeemed that they may unnecessarily obscure the essence of thedisclosure. The features of the disclosure will be more clearlyunderstood from the accompanying drawings and should not be limited bythe accompanying drawings. It is to be appreciated that all changes,equivalents, and substitutes that do not depart from the spirit andtechnical scope of the disclosure are encompassed in the disclosure.

FIG. 1 is a flowchart depicting a control method for an in-vehicle userinterface.

As shown, the control method for the in-vehicle user interface caninclude providing a vehicle window that serves as an input/output moduleengaged with at least one of a computing device, a camera and amultimedia device (10), monitoring whether a change of an air pressureat a neighboring area of the vehicle window is beyond a predeterminedthreshold range (12), sensing a touch input via the vehicle window (14),and generating, in response to the change of the air pressure and thetouch input, a control signal of the input/output module (16).

As an in-vehicle user interface, a window included in a vehicle tosecure a view of a user, a driver, or a passenger can be provided asinput/output module of at least one of a computing device, a camera, anda multimedia device. When the user, the driver, or the passenger blowsor expends breath on a vehicle window, the vehicle window can beactivated as an input/output module.

The change of the air pressure inside the vehicle (e.g., an interiorspace of the vehicle) is very slight when the windows and doors of thevehicle are closed while driving. Accordingly, when the user, thedriver, or the occupant blows his or her breath on a specific area or aspecific space, the air pressure may suddenly or rapidly change in thatarea or that space due to the breathing. When an air pressure sensor maysense this change of the air pressure, the user interface control devicecan generate a trigger signal that makes the vehicle window included inthe vehicle available as an input/output module. In order to generatethe trigger signal, the sensing range of the air pressure sensor couldbe determined within a pressure range of breathing of a person. If theair pressure sensor senses a change of air pressure beyond a level or arange of the air pressure changed by a person's breathing, it isdifficult for the user, the driver or the passenger to use his or herbreath so as to activate the vehicle window as an input/output module.

Temperature, humidity, or the like could also be used to detect theperson's breathing. However, the range of temperature or humidity thatcan be changed by the person's breathing can be different in response toan internal condition, circumstance or environment of the vehicle. Inthe case of detecting the temperature and humidity only without thechange of air pressure, erroneous detection could be increased. However,since the air pressure inside the vehicle varies less sensitively thantemperature and humidity inside the vehicle, it may be more efficient todetect the person's breathing through the change in air pressure.

However, when a user, a driver, or a passenger uses a folding fan or anelectric fan inside the vehicle, a change of air pressure caused by thefan or the electric fan may occur in a manner similar to the change ofair pressure due to a person's breath. For example, the change may beconsidered a user's input when a change of air pressure generated due toa device causing wind, such as a folding fan or an electric fan, issimilar to that caused by a person's breath. Thus, in an embodiment, inorder to distinguish the cause of the change of air pressure moreprecisely, another sensor configured to detect temperature, humidity orthe like may be used to clearly recognize a breath from the user, thedriver or the passenger, with an air pressure sensor.

On the other hand, although not shown, the control method for thein-vehicle user interface may further include activating the airpressure sensor only when a door or a window of the vehicle is closed.If the door of the vehicle is open or the window is open, changes in airpressure inside the vehicle may occur without a user's breathing. Also,the flow of air entering through an open door or window may make aslight or large change, compared with the breathing of the user, thedriver or the occupant. Therefore, when the door of the vehicle is openor the window is open, it is difficult to determine whether the changeof air pressure is caused by the user's breathing. Accordingly, it ispossible to measure the change of air pressure inside the vehicle onlywhen the doors and windows of the vehicle are closed.

An input/output module provided through window(s) of the vehicle mayinclude a touch input device and a display (or screen) for at least oneof a computing device, a camera, a multimedia device, and so on. To thisend, the window(s) is/are not only made of conventional glass, but mayinclude a (transparent) touch screen panel (TSP). Herein, the touchscreen panel (TSP) is a type of input device for recognizing a positionof a screen or transmitting recognized position to a processor or asystem when a user, a driver or a passenger presses or touches thescreen with a finger or a pen. The touch screen panel (TSP) may includea touch panel, a control circuitry, a driver program such as software,and the like.

Particularly, a vehicle window can be a touch panel. The vehicle windowmay include top and bottom films/glasses deposited with a transparentelectrode, e.g., Indium Tin Oxide (ITO). The vehicle window canrecognize a position where contact occurs or where a signal generatesdue to a change of capacitance, and transmit the position to a controlcircuitry. The control circuitry may convert an analog signaltransmitted from the vehicle window into a digital signal so that thedigital signal could be recognized in a coordinate form that could bedisplayed on the screen. The driver program may receive the digitalsignal delivered from the control circuit to resolve it in order thatthe touch panel could be utilized in accordance with an applicableapparatus or system.

On the other hand, generating the control signal (16) may includeactivating the touch input device and the display or the screen inresponse to the change of air pressure (18), and deactivating the touchinput device and the display or the screen when the touch input is notentered for a predetermined time after the touch input device and thedisplay or the screen are activated (18). In addition, the generatingthe control signal (16) may further includes generating, in response tothe change of the air pressure, a signal for resetting the touch inputdevice and the display or the screen (18).

For example, after a window of a vehicle is activated by a detectedchange of an air pressure, a touch panel included in the window cantransmit a signal or a value corresponding to a user's touch on thewindow when the user, the driver or the passenger touches the windowusing a finger or the like in an activated state of the window. On theother hand, after the window is activated, the window may be deactivatedagain if the user, the driver or the occupant does not make any inputfor a predetermined time. This may be considered a case where the user,the driver or the passenger does not want to work through the window, orwhere the power consumption is reduced.

Operation of an input/output module by a user, a driver or a passengerin the vehicle can result in various actions. In this case, the controlmethod for the in-vehicle user interface may include providing a firsttouch input button for selecting at least one of a computing device, acamera, and a multimedia device on a window in response to a change ofatmospheric pressure; and providing a second touch input button forcontrolling at least selected one among the computing device, thecamera, and the multimedia device on the corresponding window. Forexample, the first touch input button may include a menu/button forselecting a device desired to be used by a user, a driver or apassenger, while the second touch input button may include a menu/buttonfor controlling or operating the selected device.

Further, when there is some limitation of a device which a user, adriver or an occupant can use via a window of the vehicle such as aninput/output module, for example, when the window may be particularlydesigned to be used only for a specific program, an application or aspecific device equipped in the vehicle, either the first touch inputbutton or the second touch input button can be provided. When anapplication or an apparatus supports functions or services responsive toa complicated input, the first touch input button or the second touchinput button may be provided as an upper or lower level menu/button.

The control method for the in-vehicle user interface can include atleast one of engaging at least one of a computing device, a camera, anda multimedia device using a window as an input/output module with anaudio-video-navigation device equipped in the vehicle, and engaging atleast one of the computing device, the camera, and the multimedia devicewith a portable or mobile device coupled with an in-vehicle wirelesscommunication terminal. Regarding extendibility, computing devicesmounted on a vehicle at the time of manufacture of a vehicle may havelimitation. However, when interworking or engaging with an externaldevice through a wired/wireless communication terminal mounted on avehicle, the extendibility of the computing devices can be enhanced. Forexample, when a mobile device and a window are mirrored each other, auser, a driver or a passenger can use the window as an input/outputmodule instead of user interfaces of the mobile device, and can use orcontrol various programs or applications supported by the mobile devicethrough the window.

A user, a driver or an occupant in the vehicle might want to dodifferent things, but a vehicle should be designed to secure a drivingsafety. Therefore, although not shown, the control method for thein-vehicle user interface may further include deactivating a windownearby driver's seat when the vehicle is detected to be moving through awheel speed sensor. The wheel speed sensor can recognize not only thespeed of a wheel but also the movement of the wheel. Herein, themovement of the wheel can be equivalently considered the movement of thevehicle. In a moving vehicle, the driver, unlike the occupant such as apassenger, should secure a driving safety so that the window nearbydriver's seat might be forcibly deactivated when the vehicle moves orstarts to move.

FIG. 2 is a schematic diagram of a vehicle including vehicle window(s)used as a user interface.

As shown, the vehicle 2 includes at least one door 4. The door 4 caninclude a window 4A, a door opening device 4C, and a window openingdevice 4B. Herein, the window 4A securing view(s) of a user, a driver, apassenger or an occupant via a glass can include a touch screen panel(TSP). The vehicle 2 can include a control unit that can monitor andcontrol a driving state of the vehicle. The control unit may be coupledwith various sensors, operating devices, and auxiliary devices mountedon, or equipped in, the vehicle so that the control unit can transmitto, and receive from, them plural data in real time while the vehicle ismoving.

An in-vehicle user interface can provide an environment to a user, adriver, a passenger or an occupant, as a module for performingoperations (e.g., recording, manipulation, web search, and data-sharing)on window(s) equipped in the vehicle 2. The in-vehicle user interfacemay be intuitive and convenient because the user, the driver, thepassenger or the occupant can use a transparent touch screen included inthe window 4A. Since anyone in the vehicle 2 can operate anaudio-video-navigation device (AVN) or an in-vehicle electronic mountedon the vehicle 2 through a transparent touch screen included in thewindow 4A, it is possible to provide some convenient functions to allseats in the vehicle.

An atmospheric pressure sensor can be provided to detect a user's,driver's, or the passenger's breathing, and may be located on aperipheral edge of the window 4A equipped in the vehicle 2. Further,according to the embodiment, a camera may be located in an inner space(inside the vehicle 2) or on a window frame from the window 4A. Abarometer may be used as the atmospheric pressure sensor to recognizeuser's, driver's or passenger's breathing at a low cost.

For example, the barometer can recognize the air pressure change when anoccupant blows his or her breath on the window 4A of the vehicle 2. Thewindow 4A sometimes condensational may be activated as a screen inresponse to a signal generated from recognized air pressure change.Thereafter, a user's touch on the window 4A can be inputted through atouch panel operated in an electrostatic manner In addition, after thetransparent touch screen of the window is activated and the usersketches on the touch panel, the user blows his or her breath on thewindow 4A. When the barometer senses the breath, contents sketched onthe touch panel, i.e., the window 4A, may be stored and a new inputwindow can be activated. At the time, stored contents can be transmittedto a device such as a portable terminal or the like engaged with thevehicle 2 through a short-range wireless communication technology (e.g.,Wi-Fi, Bluetooth, etc.).

FIG. 3 is a schematic diagram depicting an operation of an air pressuresensor.

As shown, when an air pressure sensor is installed in the vicinity ofthe window 4A included in the door 4 of the vehicle, the air pressuresensor can recognize whether a user, a driver or an occupant blows abreath 8 in a sensing range 20 of the air pressure sensor.

Herein, the sensing range 20 of the air pressure sensor may varydepending on the position of the air pressure sensor. However, it ispossible to increase the convenience of a user, a driver, or a passengerwhen the air pressure sensor would be located at an area where the user,the driver or the passenger can blow a breath comfortably in a statewhere the user, the driver or the occupant may sit in a seat.

According to an embodiment, in order to increase user's, driver's oroccupant's intuitiveness regarding the sensing range 20 for the breath8, the air pressure sensor could be installed in close proximity tovarious control buttons (e.g., a window opening button, a door openingbutton) that may be included in the door 4.

FIG. 4 illustrates a first example of an in-vehicle user interface.Particularly, in the first example, a window in a vehicle door 4 is usedas a user interface system for a multimedia device (e.g., anaudio-video-navigation device mounted on a vehicle, a computing deviceor a multimedia device possessed by a user, a driver, or an occupant,etc.) equipped in or engaged with a vehicle.

As shown, a transparent touch screen included in the window may includea display area 22, touch input buttons 24, 26, and the like. The sizeand position of the display area 22 and the touch input buttons 24, 26may be changed. The display area 22 and the touch input buttons 24, 26may be overlapped on each other on the screen. The touch input button24, 26 may include one button or two or more buttons and may deliver atouch input of a user, a driver or a passenger (e.g., touch by user'sfinger 6).

FIG. 5 illustrates a second example of the in-vehicle user interface.Particularly, in the first example, a window in a vehicle door 4 is usedas a user interface system for a multimedia device (e.g., anaudio-video-navigation device mounted on a vehicle, a computing deviceor a multimedia device possessed by a user, a driver, or an occupant,etc.) equipped in or engaged with a vehicle.

As shown, a computing device in cooperation with a transparent touchscreen used as a user interface and included in a window of a vehiclecan provide an application creating and editing a picture 32, a text 34,a memo (not shown), etc. through touch input(s) of a user, a driver or apassenger (e.g., touch by user's finger 6). The window of the vehiclecan be used as an input/output module for recognizing the touch of theuser, the driver or the passenger (e.g., touch by user's finger 6) anddisplaying inputted contents. The window can recognize informationdelivered through a touch input button 30. At the time, data or contentscreated, recorded and stored through the window of the vehicle can betransmitted to and stored in a computing device equipped in the vehicleor a mobile device, a computing device, and the like engaged with thevehicle.

FIG. 6 illustrates a third example of the in-vehicle user interface.Particularly, the third example in the case where a window in the door 4of the vehicle is used as a user interface system for a camera apparatus40 will be described.

As shown, a transparent touch screen included in the window may includea frame (i.e., a display area 42 capable of serving as a viewfinder ofthe camera apparatus 40), which displays a photographing area of alandscape or an object seen through a window, and a shutter touch button44 for controlling a capture of the photographing area. Further, thetransparent touch screen included in the window may further include anedit button 46 for enabling editing of photographed image. Herein, theshutter touch button 44 and the edit button 46 are merely by way ofexample, and may vary depending on functions that can be providedthrough a transparent touch screen in a camera module, an application,or a computing device to be interlocked.

A user, a driver or a passenger can view outside scenery, objects, etc.through the window in the vehicle, and can activate the transparenttouch screen of the window to capture or photograph scenery, objects,etc. in the frame 42. For example, the size, position, etc. of the frame42 may be fixed based on the position and view direction of the cameraapparatus 40. Also, according to an embodiment, the size, the position,and the like of the frame 42 can be adjusted, and the camera apparatus40 can change a zoom function, a direction and an angle of lens, and thelike in response to adjusted characteristics of the frame 42.

Herein, the camera apparatus 40 may be located inside the window or in awindow frame. Further, arranged within a spatial range withoutinconvenience of a user, a driver or an occupant inside the vehicle, thecamera apparatus 40 may be detachable.

Further, it is possible to edit a photograph, including a landscape, anobject, and the like, captured by the camera apparatus 40 through theedit button 46 or the like on a transparent touch screen included in awindow. Through the window, color(s), shape(s), etc. included in thephotograph may be spoiled or stored. Further, spoiled color(s),shape(s), etc. may be used in another picture, sketch, and the likethrough touch input(s) of a user, a driver, or an occupant (e.g.,touch(es) by user's finger 6).

FIG. 7 shows a control apparatus for an in-vehicle user interface.

As shown, an in-vehicle user interface control apparatus 50 can becoupled to at least one of a mobile device 74 possessed by a user, adriver or an occupant, and an audio-video-navigation (AVN) deviceequipped in a vehicle, via a wired/wireless communication technology 70.For example, a communication unit 56 in the in-vehicle user interfacecontrol apparatus 50 may be coupled with the audio-video-navigation(AVN) device 72 through a Controller Area Network (CAN), while themobile device 74 may be coupled with the communication unit 56 via ashort-range wireless communication method.

An applicable short-range communication method can include at least oneof Bluetooth, Bluetooth Low Energy, Zigbee, Ultra Wide Band (UWB),millimeter wave (mmWave), and the like.

The vehicle user interface control apparatus 50 may include a controlunit 52 configured to provide a transparent touch screen or the likeincluded in the window 4A of the vehicle as an input/output module usedfor at least one of a computing device, a camera, and a multimediadevice, an air pressure sensor 54 configured to monitor whether a changeof an air pressure at a neighboring area of the vehicle window is beyonda predetermined threshold range, and a touch input unit 58 a configuredto receive a signal corresponding to the touch input sensed by thewindow. The control unit 52 can generate a signal capable of controllingthe input/output module, in response to at least one of the change ofthe air pressure and the touch input. In addition, the predeterminedthreshold range, which can be sensed by the air pressure sensor 54, canbe set within a pressure range of the human being's breath. Further, Thevehicle user interface control apparatus 50 may include a screenadjustment unit 58 b configured to adjust or set brightness, contrast,resolution or the like of the transparent touch screen or the likeincluded in the window 4A.

The input/output module included in the window 4A of the vehicle mayinclude a touch input device and a display or screen for at least one ofa computing device, a camera, and a multimedia device. The control unit52 can activate the touch input device and the display or the screenincluded in the vehicle window 4A, in response to the change of the airpressure delivered from the air pressure sensor 54. Further, thecontroller 52 may deactivate the touch input device, the display, or thescreen if there is no touch input from the user, the driver, or thepassenger for a predetermined time after the touch input device, thedisplay, or the screen are activated. In addition, the control unit 52in the vehicle user interface control apparatus 50 may generate a signalfor resetting the touch input device, the display, and/or the screen inresponse to the change in the air pressure.

The vehicle user interface control apparatus 50 is configured torecognize which device the transparent touch screen included in thevehicle window 4A is engaged with. For this purpose, the controller 52activates the transparent touch screen in the vehicle window 4A inresponse to the change of air pressure delivered from the air pressuresensor 54, and then receives from the user, the driver or the passengerwhich device the transparent touch screen in the vehicle window 4A isinterlocked with. For example, the control unit 52 may provide a firsttouch input button for selecting at least one of a computing device, acamera, and a multimedia device into a user, a driver or a passenger,via a transparent touch screen in the vehicle window 4A. In addition,the control unit 52 may provide a second touch input button foroperating or controlling at least one selected among the computingdevice, the camera, and the multimedia device in response to user'sinput through the first touch input button into the user, the driver orpassenger, through the transparent touch screen in the window 4A.

For example, when the camera 68 is selected through the first touchinput button, the provided second touch input button includes a framedetermining a photography area of scenery or an object through thevehicle window, a shutter button taking a photograph or a video throughthe frame, and the like. In addition, the second touch input button mayfurther include an editing button editing the photograph or the video.

Meanwhile, the computing device or the portable terminal 74 engaged withthe vehicle user interface control apparatus 50 can provide anapplication for creating, writing and editing pictures, texts, memos andetc., and store information that is displayed or recorded via thetransparent touch screen serving as an input/output module in a memory.

The vehicle user interface control device 50 can include a processingsystem that comprises at least one data processor and at least onecomputer-readable memory storing a computer program. According to anembodiment, the vehicle user interface control device 50 may beimplemented in a single circuitry chip or board.

In addition, the in-vehicle user interface control apparatus 50 may beconfigured to engage at least one of the computing device, the camera,and the multimedia device with an audio-video-navigation device mountedin the vehicle. Or, the in-vehicle user interface control apparatus 50may include a communication unit 56 configured to engage at least one ofthe computing device, the camera, and the multimedia device with amobile device 74 coupled with a wireless communication terminal equippedin the vehicle.

The vehicle user interface control apparatus 50 can be interlocked withthe vehicle sensor unit 60. For example, the vehicle sensor unit 60 mayinclude a wheel speed sensor 62 configured to sense the movement of thevehicle. When the wheel speed sensor 62 senses the movement of thevehicle, the control unit 52 may deactivate the window near driver'sseat to secure driving safety.

The air pressure sensor 54 included in the in-vehicle user interfacecontrol apparatus 50 can be engaged with the vehicle sensor unit 60. Thevehicle sensor unit 60 may include a door sensor 64 configured to detectwhether the door mounted on the vehicle is opened and the window sensor66 configured to detect whether the window is opened. Based on theinformation transmitted from the door sensor 64 and the window sensor66, the controller 82 may activate the air pressure sensor 54 only whenthe door sensor 64 informs that the door is closed and the window sensor66 informs that the window 4A is closed. This is because a malfunctionof the air pressure sensor 54 might be prevented and power consumptioncould be reduced. When the window 4A is open, the air pressure sensor 54adjacent to opened window 4A can be deactivated.

In a vehicle storing an electric energy, a battery management unit (notshown) can be engaged with the vehicle user interface control apparatus50 through the control unit 52. Further, according to an embodiment, thevehicle user interface control apparatus 50 may further include abattery management unit configured to monitor a state of charge (SOC) ofthe battery mounted inside the vehicle. The control unit 52 can receivethe state of charge (SOC) of the battery from the battery managementunit and limit or restrict activation of the window 4A when the state ofcharge (SOC) of the battery is lower than a predetermined range orlevel.

As above described, disclosed embodiments can provide a control methodor a control apparatus that allows a vehicle window to be used as a userinterface of a computing device while the driver, user or occupant usingthe vehicle is staying in the vehicle, even if the driver, user oroccupant using the vehicle does not have a separate computing device oran associated user interface.

In addition, embodiments can improve the intuitiveness of a userinterface, which is available in a vehicle, by interlocking operationsof a computing device with a behavior of a driver, a user, or apassenger using the vehicle such as his/her hands (finger) or breaths.

Further, since embodiments do not require a separate device forperforming a task required by a driver, a user, or a passenger who isnot directly related to vehicle, it is possible to enhance drivingsafety of the vehicle. Further, user's convenience can be increased.

The aforementioned embodiments are achieved by disclosure in apredetermined manner Each of the structural combination of structuralelements and features of the elements or features can be consideredselectively unless specified separately. Each of the structural elementsor features may be carried out without being combined with otherstructural elements or features. Also, some structural elements and/orfeatures may be combined with one another to constitute the embodimentsof the disclosure. The order of operations described in the embodimentsof the disclosure may be changed. Some structural elements or featuresof one embodiment may be included in another embodiment, or may bereplaced with corresponding structural elements or features of anotherembodiment. Moreover, it will be apparent that some claims referring tospecific claims may be combined with other claims referring to the otherclaims other than the specific claims to constitute the embodiment oradd new claims by means of amendment after the application is filed.

Various embodiments may be implemented using a machine-readable mediumhaving instructions stored thereon for execution by a processor toperform various methods presented herein. Examples of possiblemachine-readable mediums include HDD (Hard Disk Drive), SSD (Solid StateDisk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, a magnetic tape, afloppy disk, an optical data storage device, the other types of storagemediums presented herein, and combinations thereof. If desired, themachine-readable medium may be realized in the form of a carrier wave(for example, a transmission over the Internet).

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present disclosurewithout departing from the spirit or scope of the disclosures. Thus, itis intended that the disclosure covers the modifications and variationsof this disclosure provided they come within the scope of the appendedclaims and their equivalents.

What is claimed is:
 1. A method for controlling an in-vehicle userinterface, the method comprising: providing a vehicle window engagedwith at least one of a computing device, a camera, and a multimediadevice, wherein the vehicle window includes a touch input device and adisplay or screen used for the at least one of the computing device, thecamera, and the multimedia device; monitoring whether a change of an airpressure at a neighboring area of the vehicle window is beyond apredetermined threshold range; activating, in response to the change ofthe air pressure, the touch input device used for controlling the atleast one of the computing device, the camera, and the multimediadevice; selecting the at least one of the computing device, the camera,and the multimedia device by sensing a touch input via a first touchinput button of the touch input device on the vehicle window; generatingin response to the change of the air pressure and the touch input, acontrol signal of the touch input device; and showing, in response tothe touch input via the first touch input button, a second touch inputbutton of the touch input device on the vehicle window, the second touchinput button being used for controlling the selected one of thecomputing device, the camera, and the multimedia device, and whereingenerating the control signal comprises: deactivating the touch inputdevice and the display or the screen when the touch input is not enteredfor a predetermined time after the touch input device and the display orthe screen are activated.
 2. The method according to claim 1, whereinthe predetermined threshold range is set within a pressure range ofbreathing of a person.
 3. The method according to claim 1, whereingenerating the control signal further comprises: generating, in responseto the change of the air pressure, a signal for resetting the touchinput device and the display or the screen.
 4. The method according toclaim 1, wherein the second touch input button, when the camera isselected via the first touch input button, includes: a frame determininga photography area of scenery or an object through the vehicle window; ashutter button taking a photograph or a video through the frame; and anediting button editing the photograph or the video.
 5. The methodaccording to claim 1, further comprising: engaging the at least one ofthe computing device, the camera and the multimedia device with anaudio-video-navigation device equipped in a vehicle; and engaging the atleast one of the computing device, the camera and the multimedia devicewith a mobile device coupled with a wireless communication unit equippedin the vehicle.
 6. The method according to claim 1, further comprising:deactivating the vehicle window adjacent to driver's seat when movementof the vehicle is detected by a wheel speed sensor; and enabling an airpressure sensor detecting the change of the air pressure when a vehicledoor and the vehicle window are closed.
 7. An apparatus for controllingan in-vehicle user interface, the apparatus comprising: a controllerconfigured to provide a vehicle window engaged with at least one of acomputing device, a camera and a multimedia device, wherein the vehiclewindow includes a touch input device and a display or screen used forthe at least one of the computing device, the camera, and the multimediadevice; an air pressure sensor configured to monitor whether a change ofan air pressure at a neighboring area of the vehicle window is beyond apredetermined threshold range; and the touch input device configured toselect the at least one of the computing device, the camera, and themultimedia device by sensing a touch input via the touch input device ofthe vehicle window, wherein the controller activates the touch inputdevice and the display or the screen in response to the change of theair pressure, and deactivates the touch input device and the display orthe screen when the touch input is not entered for a predetermined timeafter the touch input device and the display or the screen areactivated, wherein the controller generates a control signal of thetouch input device in response to the change of the air pressure and thetouch input, and wherein the controller shows, in response to the touchinput via the first touch input button, a second touch input button ofthe touch input device on the vehicle window, the second touch inputbutton being used for controlling the selected one of the computingdevice, the camera, and the multimedia device.
 8. The apparatusaccording to claim 7, wherein the predetermined threshold range is setwithin a pressure range of breathing of a person.
 9. The apparatusaccording to claim 7, wherein the controller generates, in response tothe change of the air pressure, a signal for resetting the touch inputdevice and the display or the screen.
 10. The apparatus according toclaim 7, further comprising a communication unit configured to: engagethe at least one of the computing device, the camera and the multimediadevice with an audio-video-navigation device equipped in a vehicle via acontroller area network; and engage the at least one of the computingdevice, the camera and the multimedia device with a mobile device via ashort-range wireless communication method.
 11. The apparatus accordingto claim 7, wherein the air pressure sensor is arranged at a surroundingarea of the vehicle window, while the camera is arranged in a vehiclewindow frame.
 12. The apparatus according to claim 7, further comprisinga wheel speed sensor configured to detect movement of a vehicle, whereinthe controller deactivates the vehicle window adjacent to driver's seatwhen the wheel speed sensor detects movement of the vehicle.
 13. Theapparatus according to claim 7, further comprising: a door sensorconfigured to detect opening or closing of a vehicle door; and a windowsensor configured to detect opening or closing of the vehicle window,wherein the controller enables the air pressure sensor detecting thechange of the air pressure when the vehicle door and the vehicle windoware closed.